We consider the full-duplex (FD) two-way amplify-and-forward relay systemwith imperfect cancelation of loopback self-interference (SI) and investigatejoint design of relay and receive beamforming for minimizing the mean squareerror under a relay transmit power constraint. Due to loopback channelestimation error and limitation of analog-to-digital converter, the loopback SIcannot be completely canceled. Multiple antennas at the relay can help loopbackSI suppression but beamforming is required to balance between the residual SIsuppression and the desired signal transmission. Moreover, the relaybeamforming matrix should be updated every time slot because the residual SI inthe previous time slot is amplified by the current beamforming matrix and addedto the received signals from the two sources in the current time slot. Wederive the optimally balanced relay beamforming and receive beamformingmatrices in closed form based on minimum mean square error, taking into accountthe propagation of the residual loopback SI from the first to the current timeslot. We also propose beamforming design using only the channels of the mlatest time slots, not from the first time slot. Based on our numericalresults, we also identify when FD is beneficial and propose selection betweenFD and half-duplex according to signal-to-noise ratio and interference-to-noiseratio.
展开▼